We had good opportunities but unfortunately we couldn't convert them into points," France manager Raphael Ibanez said.
South Africa captain Siya Kolisi said the bench was critical.
Jonathan Danty was held up and Arendse dived over for South Africa after collecting a fortunate bounce from a speculative kick.
With barely time to breathe, South Africa regained control when Kolbe outpaced Damian Penaud with astonishing ease to collect Jesse Kriel's kick and score.
Pollard added another penalty before Ramos brought France back within a point, but South Africa defended brilliantly to keep out a late French onslaught.
Persons:
France's, Fabien Galthie, Lee Arendse, Damian de Allende, Cheslin Kolbe, Eben Etzebeth, Mannie, Handre Pollard, Antoine Dupont, scrumhalf, Les Bleus, Cyril Baille, Peato Mauvaka, Thomas Ramos, Raphael Ibanez, Siya Kolisi, Jacques Nienaber's, Dupont, Baille, Jonathan Danty, Arendse, De Allende, Mauvaka, Kolbe, Damian Penaud, Jesse Kriel's, maul, Ramos, Atonio, Nienaber, Etzebeth, Pollard, Julien Pretot, Ed Osmond
Organizations:
France, Galthie Springboks, Springboks, England, Kurt, Mauvaka, South, Thomson
Locations:
Africa, France, England, PARIS, South Africa, South